Feature/code sandbox security v3#26
Conversation
… compatibility for tests, fix decoding bug, enforce output limits, update versioning, and correct gitignore entries for logs and newline compliance.
…warnings, and improved safety controls for enterprise-grade execution behavior and user awareness
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughBumps version to 3.2.1, adds a release script, and implements a safety/execution redesign: injected ExecutionSafetyManager with AST- and pattern-based blocking, centralized sandboxing, longer timeouts, command normalization, subprocess/PackageManager changes, extensive tests and docs updates. Changes
Sequence Diagram(s)sequenceDiagram
actor User
participant CLI as Interpreter (CLI)
participant Lib as InterpreterLib
participant Safety as ExecutionSafetyManager
participant Code as CodeInterpreter
participant Proc as Process/Subprocess
User->>CLI: submit code/command [--unsafe optional]
CLI->>Lib: start session (unsafe_mode)
Lib->>Safety: instantiate(unsafe_mode)
Lib->>Code: create(safety_manager)
User->>Lib: send code/command
Lib->>Code: execute_code(code)
Code->>Safety: assess_execution(code, "code")
alt Decision.allowed == True
Code->>Lib: maybe request confirmation
alt User approves (if prompted)
Code->>Proc: run in sandbox or direct
Proc-->>Code: output / error
else User declines
Code-->>Lib: cancelled
end
else Decision.allowed == False
Safety-->>Code: Decision(allowed=False, reasons)
Code-->>Lib: report "Dangerous operation blocked"
end
Code-->>Lib: (output, error, sandbox_context)
Lib-->>User: display results / cleanup
sequenceDiagram
participant CodeInterp as CodeInterpreter
participant SafetyMgr as ExecutionSafetyManager
participant AST as AST Analyzer
participant Pattern as Pattern/Heuristics
CodeInterp->>SafetyMgr: assess_execution(code, mode)
rect rgba(150,50,200,0.5)
SafetyMgr->>AST: _ast_check(code)
AST-->>SafetyMgr: ast_reasons
SafetyMgr->>Pattern: check path/write/delete patterns
Pattern-->>SafetyMgr: pattern_reasons
alt any violations
SafetyMgr-->>CodeInterp: Decision(allowed=False, reasons=[...])
else no violations
SafetyMgr-->>CodeInterp: Decision(allowed=True, reasons=[])
end
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Poem
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Note Unit test generation is a public access feature. Expect some limitations and changes as we gather feedback and continue to improve it. Generating unit tests... This may take up to 20 minutes. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 00b9466419
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| delete_patterns = [ | ||
| r"\bunlink\b", | ||
| r"\bunlinksync\b", | ||
| r"\bremove\(", | ||
| r"\bos\.remove\b", |
There was a problem hiding this comment.
Reinstate blocking for non-delete destructive commands
This new safety gate only enumerates delete-related patterns, so commands such as shutdown, reboot, mkfs, format, and diskpart are no longer rejected by assess_execution in safe mode. As a result, dangerous system-level operations can pass through command/script checks (and can run automatically when execution is pre-approved), which is a significant safety regression compared with the prior pattern set.
Useful? React with 👍 / 👎.
| if not getattr(self, "UNSAFE_EXECUTION", False): | ||
| if any(k in command for k in ["unlink(", "remove(", "rmtree", "del ", "rm "]): | ||
| return None, "Blocked: destructive operation (LLM safety)." |
There was a problem hiding this comment.
Wire unsafe flag into command destructive-op guard
The destructive-command guard is keyed off self.UNSAFE_EXECUTION, but CodeInterpreter never defines that attribute. Because getattr(..., False) always yields False, this block always runs and rejects rm/del/remove command strings even when the interpreter is launched with --unsafe. That breaks the documented unsafe-mode behavior for command mode.
Useful? React with 👍 / 👎.
| @@ -0,0 +1,77 @@ | |||
| ```bash | |||
There was a problem hiding this comment.
Remove markdown fence from build_release.sh
Starting the script with a literal markdown fence causes Bash to treat the body as command-substitution text (closed by the trailing fence) instead of normal script statements. Running bash build_release.sh then fails by trying to execute the captured output as a command, so the release helper is not reliably runnable in its checked-in form.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Actionable comments posted: 13
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
tests/test_interpreter.py (1)
206-210:⚠️ Potential issue | 🟡 MinorTest expectation change: JS relative unlink now blocked.
Similar to the Python test, this test name says "allows" but now asserts
False. Consider renaming totest_safety_manager_blocks_js_unlink_on_relative_pathfor clarity.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/test_interpreter.py` around lines 206 - 210, The test name test_safety_manager_allows_js_unlink_on_relative_path is misleading because the asserted behavior is now blocking; rename the test to test_safety_manager_blocks_js_unlink_on_relative_path and update any references to that function name (test runner docs or imports) to match; ensure the test still constructs ExecutionSafetyManager(), calls assess_execution(code, "code"), and asserts decision.allowed is False so the implementation references (ExecutionSafetyManager and assess_execution) remain correct.libs/code_interpreter.py (1)
91-103:⚠️ Potential issue | 🟠 MajorTimeout cleanup should terminate the whole command process group.
On POSIX,
_get_subprocess_security_kwargs()starts commands in a new session, but the timeout path only kills the direct child. Any grandchildren can keep running after the interpreter reports a timeout. Mirror theos.killpg(...)handling you already use inexecute_code()so timed-out commands cannot leave background work behind.💡 Possible fix
except subprocess.TimeoutExpired: if process: - process.kill() + try: + if os.name != "nt": + os.killpg(os.getpgid(process.pid), signal.SIGKILL) + else: + process.kill() + except Exception: + process.kill() + process.communicate() return None, "Execution timed out."Also applies to: 616-619
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/code_interpreter.py` around lines 91 - 103, The timeout cleanup currently only terminates the direct child process, which can leave grandchildren running; update the timeout handling in execute_code (the code path that catches subprocess.TimeoutExpired) to mirror the POSIX process-group termination logic used in _get_subprocess_security_kwargs: when os.name != "nt" and the subprocess was started with start_new_session=True, send os.killpg(os.getpgid(proc.pid), signal.SIGKILL) (or equivalent) to kill the entire process group, falling back to proc.kill() on Windows or if group kill fails; ensure you reference the same kwargs generated by _get_subprocess_security_kwargs to detect whether a new session/process group was created and document this behavior in the timeout branch.
🧹 Nitpick comments (8)
README.md (1)
150-153: Duplicate--unsafeflag example.The
--unsafeflag usage is documented identically at lines 150-153 and again at lines 169-172. Consider removing one instance to avoid redundancy.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@README.md` around lines 150 - 153, The README contains a duplicated usage example for the --unsafe flag ("interpreter --unsafe") — remove one of the two identical blocks so the flag is documented only once; locate the duplicate example lines showing the `--unsafe` flag (the code block containing "interpreter --unsafe") and delete the redundant occurrence, keeping the clearer/earlier instance and validating surrounding headings or anchors remain consistent..gitignore (1)
10-11: Redundant log directory ignore patterns.Lines 10-11 add
/logs/andlogs/*, but line 34 already haslogs/. These patterns overlap:
/logs/ignores root-level logs directorylogs/*ignores contents of any logs directorylogs/(line 34) ignores any logs directory anywhereConsider consolidating to a single pattern for clarity.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.gitignore around lines 10 - 11, The .gitignore contains overlapping patterns (/logs/, logs/*, and logs/)—remove the redundant entries and consolidate to a single clear pattern (e.g., keep only "logs/") so that only one ignore rule covers the logs directory; update/remove the duplicate '/logs/' and 'logs/*' entries and ensure the remaining pattern matches your intended scope.build_release.sh (4)
7-13: Useread -rto prevent backslash mangling.The
readcommand without-rwill interpret backslashes as escape characters, which could cause unexpected behavior if user input contains them.♻️ Proposed fix
confirm() { - read -p "⚠️ $1 (y/N): " choice + read -rp "⚠️ $1 (y/N): " choice case "$choice" in y|Y ) return 0 ;; * ) echo "❌ Skipped: $1"; return 1 ;; esac }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@build_release.sh` around lines 7 - 13, The confirm() helper uses read without -r which allows backslashes in user input to be treated as escape characters; update the read invocation inside confirm() (the line that prompts and assigns to choice) to use read -r -p so backslashes are not mangled (preserve prompt and return behavior of confirm()).
45-45: Quote command substitution to prevent word splitting.The unquoted
$(git describe ...)could cause issues if the tag contains spaces or special characters. Additionally, if no tags exist, the command fails silently and the range becomes..HEADwhich may produce unexpected results.♻️ Proposed fix
-COMMITS=$(git log --pretty=format:"- %s" $(git describe --tags --abbrev=0 2>/dev/null)..HEAD) +LAST_TAG=$(git describe --tags --abbrev=0 2>/dev/null || echo "") +if [ -n "$LAST_TAG" ]; then + COMMITS=$(git log --pretty=format:"- %s" "${LAST_TAG}..HEAD") +else + COMMITS=$(git log --pretty=format:"- %s") +fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@build_release.sh` at line 45, The COMMITS assignment uses an unquoted command substitution with git describe --tags --abbrev=0 which can break on tags with spaces and fails silently when no tags exist; fix by first capturing the tag into a variable from git describe --tags --abbrev=0 2>/dev/null, check if that variable is empty and set the commit range to either "$TAG..HEAD" when TAG is present or to "HEAD" (or an empty range as desired) when absent, and then use that quoted range in the git log --pretty=format:"- %s" invocation so the substitution is properly quoted and robust; update the code that assigns COMMITS to reference the new variable and the quoted range.
57-59:git add .stages all files including unintended changes.Using
git add .may accidentally stage untracked files or local modifications not related to the release. Consider staging only the specific files modified by this script.♻️ Proposed fix
if confirm "Commit changes?"; then - git add . + git add "$VERSION_FILE" "$CHANGELOG_FILE" git commit -m "Release $NEW_VERSION" || echo "⚠️ Nothing to commit" fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@build_release.sh` around lines 57 - 59, The script currently uses git add . which can stage unintended/untracked files; in build_release.sh (around the confirm/commit block) replace git add . with staging only the files the script modifies—create an explicit array (e.g., RELEASE_FILES) containing the version file(s), changelog, and any other files the script updates, then run git add "${RELEASE_FILES[@]}"; optionally include git add -u to pick up updates to already-tracked files but avoid untracked files; keep the existing git commit -m "Release $NEW_VERSION" and confirm flow intact.
66-72: Quote variables to prevent globbing and word splitting.The
$NEW_VERSIONvariable should be quoted in git commands to handle edge cases safely.♻️ Proposed fix
if confirm "Create & push tag $NEW_VERSION?"; then - git tag $NEW_VERSION - git push origin $NEW_VERSION + git tag "$NEW_VERSION" + git push origin "$NEW_VERSION" fi if confirm "Create GitHub release?"; then - gh release create $NEW_VERSION --title "$NEW_VERSION" --generate-notes + gh release create "$NEW_VERSION" --title "$NEW_VERSION" --generate-notes fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@build_release.sh` around lines 66 - 72, The script uses the NEW_VERSION variable unquoted in command invocations which can cause globbing or word-splitting; update the confirm prompts and all command uses to quote the variable, e.g., use confirm "Create & push tag $NEW_VERSION?" -> confirm "Create & push tag $NEW_VERSION?" (keep prompt but ensure expansion is safe) and change git tag $NEW_VERSION, git push origin $NEW_VERSION, and gh release create $NEW_VERSION --title "$NEW_VERSION" --generate-notes to use "$NEW_VERSION" (and also quote the --title value) so the unique symbols to change are the git tag invocation, git push origin invocation, and gh release create invocation that reference NEW_VERSION.tests/test_interpreter.py (1)
538-546: Test names don't match updated behavior.
test_allows_relative_del_commandandtest_allows_del_without_pathnow assertFalse(blocked), but the names and docstrings still say "should be blocked" which is correct in the docstring but the method name says "allows". Consider renaming for consistency.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/test_interpreter.py` around lines 538 - 546, Rename the misnamed test functions to reflect the updated behavior: change test_allows_relative_del_command to test_disallows_relative_del_command and test_allows_del_without_path to test_disallows_del_without_path (and update their docstrings if you want explicit consistency), so the function names match the assertions that the safety_manager.assess_execution("del *.txt", "command") and assess_execution("del notes.txt", "command") expect decision.allowed to be False; ensure the test references (function names) are updated wherever they're used.CHANGELOG.md (1)
5-6: Duplicate changelog entries for execution architecture.Lines 5 and 6 both describe execution architecture changes with overlapping content. Consider consolidating into a single, clearer entry.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@CHANGELOG.md` around lines 5 - 6, CHANGELOG.md contains duplicate entries describing the same execution architecture changes (the two bullets shown); consolidate them into a single, clear changelog entry by merging the overlapping text, keeping key points (python-first model, sandboxing/safety controls, bash test compatibility, decoding bug fix, output limits, versioning, gitignore/log newline fixes) and removing the redundant line so only one unified bullet remains that succinctly covers all these items.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CHANGELOG.md`:
- Line 4: The CHANGELOG entry contains a typo: the word "Exectution" should be
"Execution"; update the text in CHANGELOG.md so the phrase reads "Update the
Sandbox and Code Execution" (replace the incorrect token "Exectution" with
"Execution") to correct the spelling.
- Line 8: Update the changelog entry string "- Removed /shell command and added
Code Exeuction safety" to correct the typo "Exeuction" to "Execution" so it
reads "- Removed /shell command and added Code Execution safety"; locate and
edit that exact line in CHANGELOG.md to replace the misspelled word.
In `@interpreter.py`:
- Line 29: Update the INTERPRETER_VERSION constant in interpreter.py (symbol
INTERPRETER_VERSION) so it exactly matches the project's VERSION file; change
the value from "3.2.0" to "v3.2.1" (or to "3.2.1" if the codebase convention
omits the leading "v") to make the constant consistent with the VERSION file.
In `@libs/code_interpreter.py`:
- Around line 83-89: The CodeInterpreter constructor never sets
self.UNSAFE_EXECUTION, so the interpreter always enforces safe-mode regardless
of the injected ExecutionSafetyManager; update __init__ in CodeInterpreter to
set self.UNSAFE_EXECUTION = safety_manager.unsafe_mode (or False if
safety_manager is None) after assigning self.safety_manager so the
instance-level guard matches the injected manager, and also apply the same fix
to the other initialization path referenced (the block around where
self.safety_manager is set at the later occurrence) so both
constructors/initializers read the manager's unsafe_mode into
self.UNSAFE_EXECUTION; reference the CodeInterpreter class, its __init__, the
self.UNSAFE_EXECUTION attribute, and the ExecutionSafetyManager/unsafe_mode
property when making the change.
In `@libs/interpreter_lib.py`:
- Around line 876-878: The new success message is being printed even when the
user declined execution; update execute_code() (and the branches that set
self._last_execution_approved) so the "Execution completed successfully." path
only runs when self._last_execution_approved is True (or return a distinct
"skipped" status) instead of checking execute.lower() == 'y' directly; ensure
any else branches that currently print success are gated by
self._last_execution_approved (or handle a skipped state) — relevant symbols:
execute_code(), self._last_execution_approved, and the "Execution completed
successfully." print paths (also fix identical cases around the other
occurrences noted).
- Around line 338-352: _execute_generated_output currently always cleans up the
sandbox in the finally block via
safety_manager.cleanup_sandbox_context(sandbox_context), which deletes
sandbox_context.cwd before the caller (e.g., interpreter_main) can access
outputs like graph.png/chart.png/table.md; change the flow so artifacts are
collected or exported before cleanup: either (A) have _execute_generated_output
call safety_manager.export_artifacts(sandbox_context, artifact_list_or_dest) to
copy allowed outputs out of sandbox and only then call cleanup, or (B) stop
performing cleanup in the finally and instead return the sandbox_context (or a
list of generated artifact paths) alongside the output/error so the caller
(interpreter_main) can retrieve files and then call
safety_manager.cleanup_sandbox_context(sandbox_context) when done; update the
signature/return of _execute_generated_output and downstream call sites
accordingly to ensure cleanup happens after artifact collection.
In `@libs/package_manager.py`:
- Around line 18-31: In _run_command, avoid passing a list with shell=True: when
os.name == 'nt' convert args to a single command string using
subprocess.list2cmdline(args) (or otherwise join to a safe string) and pass that
string to subprocess.check_call with shell=True; tighten the allowlist regex
(safe_pattern) to remove the '/' character so forward slashes are disallowed;
and improve the ValueError to include the specific offending argument (e.g.,
raise ValueError(f"Unsafe command argument: {arg}")) so the failing argument is
visible in logs.
In `@libs/safety_manager.py`:
- Around line 111-113: When unsafe_mode is true, don't short-circuit and return
Decision(True, []) — instead call the same dangerous-operation classifier used
elsewhere (is_dangerous_operation) from assess_execution() and propagate its
result (warnings/flags) into the returned Decision so shell execution and
AST-only hazards are still detected; update assess_execution() to invoke
is_dangerous_operation(...) and return Decision(True, warnings) (or include any
confirmation-required markers it produces) rather than an empty-warning
Decision, and mirror the same behavior for the other early-return block
referenced in the 210-236 region so both paths reuse the classifier.
In `@README.md`:
- Line 357: README's displayed version string "Current version: **3.2.0**" is
out of sync with the project's VERSION file ("v3.2.1"); update the README entry
to match the canonical VERSION value (e.g., "Current version: **3.2.1**") and
ensure this matches whatever version literal was adjusted in interpreter.py
(look for the version constant or variable in interpreter.py) so documentation
and code are consistent.
In `@tests/test_interpreter.py`:
- Line 173: The test has a logic bug in the assertion: remove the outer
generator so self.assertTrue checks the result of any(...) directly; replace the
current nested expression with a single assertion calling
self.assertTrue(any("blocked" in r.lower() for r in decision.reasons)) to verify
at least one reason contains "blocked" (referencing decision.reasons and the
test's self.assertTrue).
- Around line 175-179: The test name and assertion are inconsistent: update the
test to reflect the new SAFE MODE policy that blocks relative file deletes by
either renaming test_safety_manager_allows_relative_file_delete to something
like test_safety_manager_blocks_relative_file_delete and keep the assertion that
ExecutionSafetyManager().assess_execution(code, "code").allowed is False, or if
behavior was unintended, change the assertion to True; specifically locate the
test function test_safety_manager_allows_relative_file_delete and ensure the
expectation for decision.allowed matches the intended policy for
ExecutionSafetyManager.assess_execution when code contains
os.remove('temp.txt').
In `@VERSION`:
- Line 1: The VERSION file and interpreter.py disagree: VERSION contains
"v3.2.1" while interpreter.py defines INTERPRETER_VERSION = "3.2.0"; fix by
making them identical—either update the INTERPRETER_VERSION constant in
interpreter.py to match the VERSION file (use "v3.2.1" or "3.2.1" depending on
the codebase convention) or change the VERSION file to "3.2.0"; update
INTERPRETER_VERSION in interpreter.py (search for the symbol
INTERPRETER_VERSION) and ensure tests (e.g.,
test_version_file_matches_interpreter_version_constant) pass after the change.
---
Outside diff comments:
In `@libs/code_interpreter.py`:
- Around line 91-103: The timeout cleanup currently only terminates the direct
child process, which can leave grandchildren running; update the timeout
handling in execute_code (the code path that catches subprocess.TimeoutExpired)
to mirror the POSIX process-group termination logic used in
_get_subprocess_security_kwargs: when os.name != "nt" and the subprocess was
started with start_new_session=True, send os.killpg(os.getpgid(proc.pid),
signal.SIGKILL) (or equivalent) to kill the entire process group, falling back
to proc.kill() on Windows or if group kill fails; ensure you reference the same
kwargs generated by _get_subprocess_security_kwargs to detect whether a new
session/process group was created and document this behavior in the timeout
branch.
In `@tests/test_interpreter.py`:
- Around line 206-210: The test name
test_safety_manager_allows_js_unlink_on_relative_path is misleading because the
asserted behavior is now blocking; rename the test to
test_safety_manager_blocks_js_unlink_on_relative_path and update any references
to that function name (test runner docs or imports) to match; ensure the test
still constructs ExecutionSafetyManager(), calls assess_execution(code, "code"),
and asserts decision.allowed is False so the implementation references
(ExecutionSafetyManager and assess_execution) remain correct.
---
Nitpick comments:
In @.gitignore:
- Around line 10-11: The .gitignore contains overlapping patterns (/logs/,
logs/*, and logs/)—remove the redundant entries and consolidate to a single
clear pattern (e.g., keep only "logs/") so that only one ignore rule covers the
logs directory; update/remove the duplicate '/logs/' and 'logs/*' entries and
ensure the remaining pattern matches your intended scope.
In `@build_release.sh`:
- Around line 7-13: The confirm() helper uses read without -r which allows
backslashes in user input to be treated as escape characters; update the read
invocation inside confirm() (the line that prompts and assigns to choice) to use
read -r -p so backslashes are not mangled (preserve prompt and return behavior
of confirm()).
- Line 45: The COMMITS assignment uses an unquoted command substitution with git
describe --tags --abbrev=0 which can break on tags with spaces and fails
silently when no tags exist; fix by first capturing the tag into a variable from
git describe --tags --abbrev=0 2>/dev/null, check if that variable is empty and
set the commit range to either "$TAG..HEAD" when TAG is present or to "HEAD" (or
an empty range as desired) when absent, and then use that quoted range in the
git log --pretty=format:"- %s" invocation so the substitution is properly quoted
and robust; update the code that assigns COMMITS to reference the new variable
and the quoted range.
- Around line 57-59: The script currently uses git add . which can stage
unintended/untracked files; in build_release.sh (around the confirm/commit
block) replace git add . with staging only the files the script modifies—create
an explicit array (e.g., RELEASE_FILES) containing the version file(s),
changelog, and any other files the script updates, then run git add
"${RELEASE_FILES[@]}"; optionally include git add -u to pick up updates to
already-tracked files but avoid untracked files; keep the existing git commit -m
"Release $NEW_VERSION" and confirm flow intact.
- Around line 66-72: The script uses the NEW_VERSION variable unquoted in
command invocations which can cause globbing or word-splitting; update the
confirm prompts and all command uses to quote the variable, e.g., use confirm
"Create & push tag $NEW_VERSION?" -> confirm "Create & push tag $NEW_VERSION?"
(keep prompt but ensure expansion is safe) and change git tag $NEW_VERSION, git
push origin $NEW_VERSION, and gh release create $NEW_VERSION --title
"$NEW_VERSION" --generate-notes to use "$NEW_VERSION" (and also quote the
--title value) so the unique symbols to change are the git tag invocation, git
push origin invocation, and gh release create invocation that reference
NEW_VERSION.
In `@CHANGELOG.md`:
- Around line 5-6: CHANGELOG.md contains duplicate entries describing the same
execution architecture changes (the two bullets shown); consolidate them into a
single, clear changelog entry by merging the overlapping text, keeping key
points (python-first model, sandboxing/safety controls, bash test compatibility,
decoding bug fix, output limits, versioning, gitignore/log newline fixes) and
removing the redundant line so only one unified bullet remains that succinctly
covers all these items.
In `@README.md`:
- Around line 150-153: The README contains a duplicated usage example for the
--unsafe flag ("interpreter --unsafe") — remove one of the two identical blocks
so the flag is documented only once; locate the duplicate example lines showing
the `--unsafe` flag (the code block containing "interpreter --unsafe") and
delete the redundant occurrence, keeping the clearer/earlier instance and
validating surrounding headings or anchors remain consistent.
In `@tests/test_interpreter.py`:
- Around line 538-546: Rename the misnamed test functions to reflect the updated
behavior: change test_allows_relative_del_command to
test_disallows_relative_del_command and test_allows_del_without_path to
test_disallows_del_without_path (and update their docstrings if you want
explicit consistency), so the function names match the assertions that the
safety_manager.assess_execution("del *.txt", "command") and
assess_execution("del notes.txt", "command") expect decision.allowed to be
False; ensure the test references (function names) are updated wherever they're
used.
🪄 Autofix (Beta)
✅ Autofix completed
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: f5eb5832-3cbd-432f-a99e-79d7b3f5f3ac
📒 Files selected for processing (12)
.gitignoreCHANGELOG.mdREADME.mdVERSIONbuild_release.shinterpreter.pylibs/code_interpreter.pylibs/history_manager.pylibs/interpreter_lib.pylibs/package_manager.pylibs/safety_manager.pytests/test_interpreter.py
💤 Files with no reviewable changes (1)
- libs/history_manager.py
| decision = safety_manager.assess_execution(code, "code") | ||
| self.assertFalse(decision.allowed) | ||
| self.assertTrue(any("Deleting absolute-path" in r for r in decision.reasons)) | ||
| self.assertTrue(any("blocked" in r.lower() for r in decision.reasons) for r in decision.reasons) |
There was a problem hiding this comment.
Bug: Nested any() always returns True.
The assertion has a logic error:
self.assertTrue(any("blocked" in r.lower() for r in decision.reasons) for r in decision.reasons)The outer iteration for r in decision.reasons creates a generator of booleans from the inner any(), and any() on a non-empty generator of booleans will always be True (since True and False are both truthy in a generator context—wait, actually any() returns True if any element is truthy).
The intended check is likely just the inner expression:
🐛 Proposed fix
- self.assertTrue(any("blocked" in r.lower() for r in decision.reasons) for r in decision.reasons)
+ self.assertTrue(any("blocked" in r.lower() for r in decision.reasons))🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/test_interpreter.py` at line 173, The test has a logic bug in the
assertion: remove the outer generator so self.assertTrue checks the result of
any(...) directly; replace the current nested expression with a single assertion
calling self.assertTrue(any("blocked" in r.lower() for r in decision.reasons))
to verify at least one reason contains "blocked" (referencing decision.reasons
and the test's self.assertTrue).
| def test_safety_manager_allows_relative_file_delete(self): | ||
| safety_manager = ExecutionSafetyManager() | ||
| code = r"import os\nos.remove('temp.txt')" | ||
| decision = safety_manager.assess_execution(code, "code") | ||
| self.assertTrue(decision.allowed) | ||
| self.assertFalse(decision.allowed) |
There was a problem hiding this comment.
Test expectation change: relative deletes now blocked.
The test test_safety_manager_allows_relative_file_delete now expects decision.allowed to be False. This is a behavioral change—relative file deletes like os.remove('temp.txt') are now blocked in SAFE MODE.
Ensure this aligns with the intended safety policy. The test name still says "allows" but the assertion expects blocking.
📝 Consider renaming the test
- def test_safety_manager_allows_relative_file_delete(self):
+ def test_safety_manager_blocks_relative_file_delete(self):
safety_manager = ExecutionSafetyManager()
code = r"import os\nos.remove('temp.txt')"
decision = safety_manager.assess_execution(code, "code")
self.assertFalse(decision.allowed)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def test_safety_manager_allows_relative_file_delete(self): | |
| safety_manager = ExecutionSafetyManager() | |
| code = r"import os\nos.remove('temp.txt')" | |
| decision = safety_manager.assess_execution(code, "code") | |
| self.assertTrue(decision.allowed) | |
| self.assertFalse(decision.allowed) | |
| def test_safety_manager_blocks_relative_file_delete(self): | |
| safety_manager = ExecutionSafetyManager() | |
| code = r"import os\nos.remove('temp.txt')" | |
| decision = safety_manager.assess_execution(code, "code") | |
| self.assertFalse(decision.allowed) |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/test_interpreter.py` around lines 175 - 179, The test name and
assertion are inconsistent: update the test to reflect the new SAFE MODE policy
that blocks relative file deletes by either renaming
test_safety_manager_allows_relative_file_delete to something like
test_safety_manager_blocks_relative_file_delete and keep the assertion that
ExecutionSafetyManager().assess_execution(code, "code").allowed is False, or if
behavior was unintended, change the assertion to True; specifically locate the
test function test_safety_manager_allows_relative_file_delete and ensure the
expectation for decision.allowed matches the intended policy for
ExecutionSafetyManager.assess_execution when code contains
os.remove('temp.txt').
|
Note Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it. The branch was updated while autofix was in progress. Please try again. |
|
Note Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it. Fixes Applied SuccessfullyFixed 7 file(s) based on 10 unresolved review comments. Files modified:
Commit: The changes have been pushed to the Time taken: |
|
Request timed out after 900000ms (requestId=221603f1-dc55-4101-8a5a-a648a17fdb1b) |
|
Note Unit test generation is a public access feature. Expect some limitations and changes as we gather feedback and continue to improve it. Generating unit tests... This may take up to 20 minutes. |
Fixed 7 file(s) based on 10 unresolved review comments. Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
|
✅ Unit tests committed locally. Commit: |
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
libs/code_interpreter.py (1)
543-549:⚠️ Potential issue | 🔴 CriticalScript mode is still executing Python as Bash on macOS/Linux.
libs/interpreter_lib.pynow asks the model for a Python script only, but these branches still send the result to/bin/bash. A valid script likeprint("ok")will fail immediately on both platforms.💡 Minimal fix
- if 'darwin' in os_type.lower() or 'macos' in os_type.lower(): - output, error = self._execute_script(script, shell='bash', sandbox_context=sandbox_context) - elif 'linux' in os_type.lower(): - output, error = self._execute_script(script, shell='bash', sandbox_context=sandbox_context) - elif 'windows' in os_type.lower(): + if 'darwin' in os_type.lower() or 'macos' in os_type.lower(): + output, error = self._execute_script(script, shell='python', sandbox_context=sandbox_context) + elif 'linux' in os_type.lower(): + output, error = self._execute_script(script, shell='python', sandbox_context=sandbox_context) + elif 'windows' in os_type.lower(): output, error = self._execute_script(script, shell='python', sandbox_context=sandbox_context)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/code_interpreter.py` around lines 543 - 549, The code is sending Python scripts to a bash shell on macOS/Linux; update the branches that check os_type (the 'darwin'/'macos' and 'linux' branches) to call self._execute_script(script, shell='python' or 'python3', sandbox_context=sandbox_context) instead of shell='bash' so Python code like print("ok") executes correctly; keep the windows branch unchanged and retain sandbox_context and the same output,error handling.libs/interpreter_lib.py (1)
593-600:⚠️ Potential issue | 🟠 Major
/executeis leaking safe-mode sandboxes.Line 593 now captures
sandbox_context, but this method never callscleanup_sandbox_context(). Every/executein SAFE mode leaves a temp sandbox directory behind.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/interpreter_lib.py` around lines 593 - 600, The code is leaking safe-mode sandboxes because _execute_generated_output returns sandbox_context but the caller never invokes cleanup_sandbox_context; update the handler that calls _execute_generated_output (the block using variables code_output, code_error, sandbox_context) to ensure cleanup_sandbox_context(sandbox_context) is called in all control paths (success, error, and exceptions), e.g., place the cleanup call in a finally block or ensure every branch (the if code_output, elif code_error, and any exception handlers) invokes cleanup_sandbox_context with the returned sandbox_context; reference _execute_generated_output and cleanup_sandbox_context to locate the change.tests/test_interpreter.py (1)
685-690:⚠️ Potential issue | 🟠 MajorUpdate these
_execute_generated_output()mocks to the new 3-tuple contract.The helper now returns
(output, error, sandbox_context), but both stubs still return 2-tuples. That's the unpacking failure reported at Lines 691 and 726 in CI.💡 Minimal fix
def fake_execute(snippet, os_name, force_execute=False): execute_calls.append(snippet) - return None, "Safety blocked: Absolute-path deletion is blocked." + return None, "Safety blocked: Absolute-path deletion is blocked.", None @@ - patch.object(interp, "_execute_generated_output", return_value=("Volume in drive D", None)): + patch.object(interp, "_execute_generated_output", return_value=("Volume in drive D", None, None)):Also applies to: 724-725
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/test_interpreter.py` around lines 685 - 690, The test mocks for _execute_generated_output use a 2-tuple return but the real contract is a 3-tuple (output, error, sandbox_context); update the fake_execute helper and any other stubbed returns in the test to return three values (e.g., return None, "Safety blocked: Absolute-path deletion is blocked.", {}) and adjust the other patched stub similarly so both patched side_effects/return_value provide (output, error, sandbox_context) consistent with _execute_generated_output.
♻️ Duplicate comments (4)
libs/package_manager.py (1)
23-26:⚠️ Potential issue | 🟠 MajorDon't block scoped npm package names on Windows.
_extract_javascript_package_name()returns raw module specifiers, and scoped npm packages include/. With the current allowlist, installs likenpm install@types/node`` now fail fast withValueErrorbefore npm even runs.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/package_manager.py` around lines 23 - 26, The allowlist regex in the args validation (safe_pattern) is blocking scoped npm package names (returned by _extract_javascript_package_name()) because it disallows '/', so update the validation to permit forward slashes in package specifiers (i.e., include '/' in the character class used to compile safe_pattern) while keeping the rest of the checks intact in the loop that raises ValueError for unsafe args; ensure you modify the safe_pattern definition referenced in the same function so scoped names like `@scope/pkg` pass validation.libs/safety_manager.py (2)
79-83:⚠️ Potential issue | 🟠 MajorNarrow
remove()matching to real filesystem APIs.The AST branch plus both regex lists treat any
.remove()/remove(call as host-file deletion. That blocks harmless code likeitems.remove("x")in SAFE mode and also triggers unsafe warnings for normal in-memory mutations.Also applies to: 128-143, 223-239
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/safety_manager.py` around lines 79 - 83, The code is overly broad by treating any .remove()/.unlink()/rmtree() call as filesystem deletion; update the AST check in safety_manager.py (the node.func.attr logic and the corresponding regex lists) to only flag calls when the receiver is a known filesystem API (e.g., module names like os.remove/os.unlink, shutil.rmtree, pathlib.Path.unlink/Path.rmdir or explicit Path-like objects), by verifying node.func.value is an ast.Name/ast.Attribute referencing those modules or classes (or by matching fully-qualified names) instead of matching the bare method name; apply the same tightening to the other branches (lines referenced around 128-143 and 223-239) so in-memory methods like items.remove(...) are no longer flagged.
111-116:⚠️ Potential issue | 🟠 MajorUnsafe mode still misses shell and AST-only hazards.
This branch only asks
is_dangerous_operation(), which is delete-pattern-only.subprocess.run(...),os.system(...),exec(...), orgetattr(os, "remove")end up asDecision(True, []), so the new warning/confirmation flow is skipped for them.Also applies to: 213-239
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/safety_manager.py` around lines 111 - 116, The unsafe_mode branch only calls is_dangerous_operation(code) and returns Decision(True, warnings) without checking shell/AST-only hazards; update the unsafe_mode handling to reuse the full hazard detection used elsewhere (e.g., call the same analyzer method that detects subprocess.run/os.system/exec/getattr patterns — likely named detect_hazards, analyze_code_safety, or the method used in the other branch) so it accumulates warnings for shell/AST-only issues as well and then returns Decision(True, warnings); reference and invoke the existing detection helpers (is_dangerous_operation plus the broader hazard detector used in the safe-confirmation flow) instead of only is_dangerous_operation.tests/test_interpreter.py (1)
173-173:⚠️ Potential issue | 🟠 MajorThis assertion currently always passes.
assertTrue(<generator>)only checks that the generator object exists. Drop the outer generator so the test actually inspectsdecision.reasons.💡 Minimal fix
- self.assertTrue(any("blocked" in r.lower() for r in decision.reasons) for r in decision.reasons) + self.assertTrue(any("blocked" in r.lower() for r in decision.reasons))🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/test_interpreter.py` at line 173, The current assertion wraps a generator and always passes; change the assertion so it evaluates a boolean by calling any over decision.reasons instead of passing a generator object—i.e., replace the outer generator usage in the self.assertTrue call so it directly uses any("blocked" in r.lower() for r in decision.reasons) to check that at least one reason contains "blocked".
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@libs/safety_manager.py`:
- Around line 162-198: is_path_access currently only detects Windows
drive-letter paths so POSIX absolute paths like '/tmp/out.txt' bypass the
read-only checks; update detection to treat POSIX absolute paths as path access
(e.g., change is_path_access logic to also detect literal strings starting with
'/' or use a regex that matches either drive-letter paths or quoted strings
beginning with '/'), and additionally when scanning open_calls ensure you
inspect the first argument literal to see if it is an absolute POSIX path and
treat that as path access; enforce the same blocking behavior (return
Decision(False, ...)) for writes/deletes regardless of sandbox_context.cwd when
an absolute path is detected.
In `@tests/test_interpreter.py`:
- Around line 1781-1784: The test assertion is expecting a list but
PackageManager._run_command() now passes a single command string on Windows;
update the test to assert the platform-specific call shape: use the same
platform check the implementation uses (e.g., os.name == "nt" or
sys.platform.startswith("win")) and assert
mock_call.assert_called_once_with("pip install requests", shell=True) on Windows
and with ["pip", "install", "requests"] on non-Windows, referencing
PackageManager._run_command and the test block that patches
subprocess.check_call.
---
Outside diff comments:
In `@libs/code_interpreter.py`:
- Around line 543-549: The code is sending Python scripts to a bash shell on
macOS/Linux; update the branches that check os_type (the 'darwin'/'macos' and
'linux' branches) to call self._execute_script(script, shell='python' or
'python3', sandbox_context=sandbox_context) instead of shell='bash' so Python
code like print("ok") executes correctly; keep the windows branch unchanged and
retain sandbox_context and the same output,error handling.
In `@libs/interpreter_lib.py`:
- Around line 593-600: The code is leaking safe-mode sandboxes because
_execute_generated_output returns sandbox_context but the caller never invokes
cleanup_sandbox_context; update the handler that calls _execute_generated_output
(the block using variables code_output, code_error, sandbox_context) to ensure
cleanup_sandbox_context(sandbox_context) is called in all control paths
(success, error, and exceptions), e.g., place the cleanup call in a finally
block or ensure every branch (the if code_output, elif code_error, and any
exception handlers) invokes cleanup_sandbox_context with the returned
sandbox_context; reference _execute_generated_output and cleanup_sandbox_context
to locate the change.
In `@tests/test_interpreter.py`:
- Around line 685-690: The test mocks for _execute_generated_output use a
2-tuple return but the real contract is a 3-tuple (output, error,
sandbox_context); update the fake_execute helper and any other stubbed returns
in the test to return three values (e.g., return None, "Safety blocked:
Absolute-path deletion is blocked.", {}) and adjust the other patched stub
similarly so both patched side_effects/return_value provide (output, error,
sandbox_context) consistent with _execute_generated_output.
---
Duplicate comments:
In `@libs/package_manager.py`:
- Around line 23-26: The allowlist regex in the args validation (safe_pattern)
is blocking scoped npm package names (returned by
_extract_javascript_package_name()) because it disallows '/', so update the
validation to permit forward slashes in package specifiers (i.e., include '/' in
the character class used to compile safe_pattern) while keeping the rest of the
checks intact in the loop that raises ValueError for unsafe args; ensure you
modify the safe_pattern definition referenced in the same function so scoped
names like `@scope/pkg` pass validation.
In `@libs/safety_manager.py`:
- Around line 79-83: The code is overly broad by treating any
.remove()/.unlink()/rmtree() call as filesystem deletion; update the AST check
in safety_manager.py (the node.func.attr logic and the corresponding regex
lists) to only flag calls when the receiver is a known filesystem API (e.g.,
module names like os.remove/os.unlink, shutil.rmtree,
pathlib.Path.unlink/Path.rmdir or explicit Path-like objects), by verifying
node.func.value is an ast.Name/ast.Attribute referencing those modules or
classes (or by matching fully-qualified names) instead of matching the bare
method name; apply the same tightening to the other branches (lines referenced
around 128-143 and 223-239) so in-memory methods like items.remove(...) are no
longer flagged.
- Around line 111-116: The unsafe_mode branch only calls
is_dangerous_operation(code) and returns Decision(True, warnings) without
checking shell/AST-only hazards; update the unsafe_mode handling to reuse the
full hazard detection used elsewhere (e.g., call the same analyzer method that
detects subprocess.run/os.system/exec/getattr patterns — likely named
detect_hazards, analyze_code_safety, or the method used in the other branch) so
it accumulates warnings for shell/AST-only issues as well and then returns
Decision(True, warnings); reference and invoke the existing detection helpers
(is_dangerous_operation plus the broader hazard detector used in the
safe-confirmation flow) instead of only is_dangerous_operation.
In `@tests/test_interpreter.py`:
- Line 173: The current assertion wraps a generator and always passes; change
the assertion so it evaluates a boolean by calling any over decision.reasons
instead of passing a generator object—i.e., replace the outer generator usage in
the self.assertTrue call so it directly uses any("blocked" in r.lower() for r in
decision.reasons) to check that at least one reason contains "blocked".
🪄 Autofix (Beta)
✅ Autofix completed
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ad57921e-a623-4b9d-ad10-7b41303491f1
📒 Files selected for processing (9)
CHANGELOG.mdREADME.mdVERSIONinterpreter.pylibs/code_interpreter.pylibs/interpreter_lib.pylibs/package_manager.pylibs/safety_manager.pytests/test_interpreter.py
✅ Files skipped from review due to trivial changes (1)
- VERSION
🚧 Files skipped from review as they are similar to previous changes (2)
- interpreter.py
- README.md
|
Note Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it. Fixes Applied SuccessfullyFixed 2 file(s) based on 2 unresolved review comments. Files modified:
Commit: The changes have been pushed to the Time taken: |
Fixed 2 file(s) based on 2 unresolved review comments. Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
…_script Bug #1 — _execute_script bash branch used SIGTERM; outer except block lacked killpg. Both now use SIGKILL via os.killpg on POSIX. Bug #4 — execute_script routed macOS/Linux to bash regardless of content. Python code (import / def / class / print / for / if / while) is now detected and dispatched to shell='python' so LLM-generated Python no longer crashes in bash."
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (3)
libs/safety_manager.py (2)
111-116:⚠️ Potential issue | 🟠 MajorUnsafe mode still misses shell and AST-only hazards.
unsafe_modeonly consults the delete-only regex helper before returningDecision(True, warnings).subprocess.run(...),os.system(...),eval(...), and obfuscated deletes caught by_ast_check()now execute with no warning/confirmation marker.Also applies to: 121-157, 229-255
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/safety_manager.py` around lines 111 - 116, The unsafe_mode branch currently only calls is_dangerous_operation(code) before returning Decision(True, warnings), so shell invocations (subprocess.run, os.system), eval/exec, and AST-detected hazards are allowed without warnings; update the unsafe_mode handling in the safety manager to also run the full suite of checks (including _ast_check(code), the shell/command pattern checks, and eval/exec detection) and append corresponding warning messages before returning Decision(True, warnings), ensuring functions/methods like is_dangerous_operation, _ast_check, and any shell/eval detection helpers are invoked and their results added to warnings so unsafe_mode still surfaces these hazards even while permitting execution.
80-89:⚠️ Potential issue | 🟠 MajorNarrow
remove()detection to filesystem APIs.These branches still treat any
.remove()/remove(call as destructive I/O, so benign code likeitems.remove("x")is blocked in SAFE mode and warned in UNSAFE mode. Please scope this to filesystem-specific owners/imports (os.remove,Path.unlink,shutil.rmtree, JSfs.unlink*, etc.) instead of matching the method name alone.Also applies to: 128-143, 239-255
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/safety_manager.py` around lines 80 - 89, The current checks in safety_manager.py treat any call named "remove"/"unlink"/"rmtree" as destructive; update the AST checks so they only match filesystem APIs. Concretely, in the ast.Attribute branch that inspects node.func.attr and in the getattr branch that inspects node.args[1], add a caller/owner check: for ast.Attribute ensure node.func.value refers to known filesystem modules/classes (e.g., ast.Name id in {"os","shutil"}, or an attribute chain like ast.Attribute with value ast.Name id "pathlib" or value ast.Name id "Path" / class instance checks) and only then treat node.func.attr in {"remove","unlink","rmtree"} as dangerous; for the getattr obfuscation branch ensure the first arg (node.args[0]) or its attribute owner resolves to the same filesystem owners before appending "AST: obfuscated deletion blocked." Update both the direct-call (node.func.attr) and getattr (node.args[1].value) checks referenced in this diff (and analogous checks at the other locations mentioned) to use the same owner-scoping logic.tests/test_interpreter.py (1)
173-173:⚠️ Potential issue | 🟡 MinorThis assertion always passes.
assertTrue(...)is receiving a generator object, not the result ofany(), so this test will stay green even when none of the reasons contains"blocked".Suggested fix
- self.assertTrue(any("blocked" in r.lower() for r in decision.reasons) for r in decision.reasons) + self.assertTrue(any("blocked" in r.lower() for r in decision.reasons))🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/test_interpreter.py` at line 173, The assertion mistakenly passes a generator object to assertTrue; change the assertion to evaluate the any() result by calling it directly — e.g. replace the line that currently reads self.assertTrue(any("blocked" in r.lower() for r in decision.reasons) for r in decision.reasons) with self.assertTrue(any("blocked" in r.lower() for r in decision.reasons)) (optionally include a failure message), referencing the test's local variable decision and its decision.reasons sequence.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@libs/code_interpreter.py`:
- Around line 67-73: The regex _PYTHON_CODE_PATTERNS used by execute_script is
overmatching shell constructs (e.g., "for ...; do" and "while ...; do") and
under-matching simple Python like "x = 1"; replace the heuristic with a
lightweight parse-based check: attempt to ast.parse() the input (in the same
function that currently uses _PYTHON_CODE_PATTERNS) and treat the input as
Python if ast.parse succeeds without SyntaxError, otherwise fall back to shell
routing; keep the regex only as a secondary heuristic if you must, but prefer
the ast.parse approach to reliably distinguish Python vs shell in execute_script
and remove or de-prioritize _PYTHON_CODE_PATTERNS usage.
- Around line 654-657: In execute_command's TimeoutExpired handler, replace the
current process.kill() call with a call to _kill_process_group(process) to
ensure the entire session/process group started via
_get_subprocess_security_kwargs(start_new_session=True) is terminated; update
the except subprocess.TimeoutExpired block to call _kill_process_group(process)
(and preserve the existing check for process) so child processes are cleaned up
correctly and the function still returns the same timeout tuple.
In `@libs/safety_manager.py`:
- Around line 185-204: The current write-detection logic (open_calls,
call_lower, code_lower, returning Decision(False,...)) is too fragile—update it
to robustly catch binary and combined modes and filesystem write APIs: extend
literal-mode checks to detect 'wb','ab','xb' and any mode= keyword values, add
pathlib method names like write_text, write_bytes, write_bytes, write_text and
generic Path().write(...) patterns, and include common JS/Node APIs (writeFile,
writeFileSync, appendFile, appendFileSync, fs.write) in the checks; ideally
replace the ad-hoc regex/substr approach with language-aware parsing (use
Python's ast to inspect Call nodes and their args/keywords for open() mode and
attribute calls, and add a simple token/regex for JS patterns) so that functions
like open_calls, call_lower and code_lower use AST-based detection and still
return Decision(False, ["Write blocked (read-only mode)."]) for any confirmed
write operation.
---
Duplicate comments:
In `@libs/safety_manager.py`:
- Around line 111-116: The unsafe_mode branch currently only calls
is_dangerous_operation(code) before returning Decision(True, warnings), so shell
invocations (subprocess.run, os.system), eval/exec, and AST-detected hazards are
allowed without warnings; update the unsafe_mode handling in the safety manager
to also run the full suite of checks (including _ast_check(code), the
shell/command pattern checks, and eval/exec detection) and append corresponding
warning messages before returning Decision(True, warnings), ensuring
functions/methods like is_dangerous_operation, _ast_check, and any shell/eval
detection helpers are invoked and their results added to warnings so unsafe_mode
still surfaces these hazards even while permitting execution.
- Around line 80-89: The current checks in safety_manager.py treat any call
named "remove"/"unlink"/"rmtree" as destructive; update the AST checks so they
only match filesystem APIs. Concretely, in the ast.Attribute branch that
inspects node.func.attr and in the getattr branch that inspects node.args[1],
add a caller/owner check: for ast.Attribute ensure node.func.value refers to
known filesystem modules/classes (e.g., ast.Name id in {"os","shutil"}, or an
attribute chain like ast.Attribute with value ast.Name id "pathlib" or value
ast.Name id "Path" / class instance checks) and only then treat node.func.attr
in {"remove","unlink","rmtree"} as dangerous; for the getattr obfuscation branch
ensure the first arg (node.args[0]) or its attribute owner resolves to the same
filesystem owners before appending "AST: obfuscated deletion blocked." Update
both the direct-call (node.func.attr) and getattr (node.args[1].value) checks
referenced in this diff (and analogous checks at the other locations mentioned)
to use the same owner-scoping logic.
In `@tests/test_interpreter.py`:
- Line 173: The assertion mistakenly passes a generator object to assertTrue;
change the assertion to evaluate the any() result by calling it directly — e.g.
replace the line that currently reads self.assertTrue(any("blocked" in r.lower()
for r in decision.reasons) for r in decision.reasons) with
self.assertTrue(any("blocked" in r.lower() for r in decision.reasons))
(optionally include a failure message), referencing the test's local variable
decision and its decision.reasons sequence.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 62141173-9423-4d61-8d79-9787938aa030
📒 Files selected for processing (3)
libs/code_interpreter.pylibs/safety_manager.pytests/test_interpreter.py
- export_artifacts(): copies PNG/MD/CSV/TXT/HTML files out of sandbox before cleanup so callers always receive generated plots/tables - assess_execution(): extend host-filesystem guard to catch unquoted POSIX absolute paths (/etc/passwd, /tmp/x, /var/log/…) that bypassed the quoted-string check — closes Bug #5 (host-write escape)
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@libs/safety_manager.py`:
- Around line 292-306: The export logic currently falls back to os.getcwd(),
uses shutil.copy2 which follows symlinks and can overwrite destinations; change
dest_dir default to a sandboxed temp directory (e.g., tempfile.mkdtemp())
instead of os.getcwd(), skip any source that is a symlink by checking
os.path.islink(src) or using os.lstat, and avoid clobbering by generating a
collision-safe destination name in the chosen dest_dir (e.g., append a uuid or
use tempfile.NamedTemporaryFile in dest_dir to obtain a unique filename) before
copying; keep the file filter using self.ARTIFACT_EXTENSIONS and record exported
mapping under the final safe dst, and use shutil.copy2 with
follow_symlinks=False (or equivalent safe copy) to ensure symlinks are not
followed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
…passes (Bug #2) - Add 'wb', 'ab', 'xb' binary write mode patterns to open() checks - Add mode= keyword argument pattern (mode='w', mode="wb", etc.) - Add pathlib write_text() / write_bytes() detection - Add JS writeFile() / writeFileSync() detection - Add pandas/DataFrame to_csv(), to_json(), to_html() with path args - Apply write checks globally (not just inside the is_path_access block) so SAFE mode catches writes that don't use absolute paths too
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (3)
libs/safety_manager.py (3)
159-163:⚠️ Potential issue | 🟠 MajorUnsafe mode still skips warnings for shell and AST-only hazards.
assess_execution()returns from theunsafe_modebranch before the shell/AST/write checks run, andis_dangerous_operation()is still delete-only. That meanssubprocess.run(...),os.system(...),exec(...), orgetattr(os, "remove")can reach the unsafe path without the new confirmation/warning flow.Also applies to: 203-212, 284-310
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/safety_manager.py` around lines 159 - 163, The unsafe_mode branch in assess_execution prematurely returns a Decision and thus skips the shell/AST/write hazard checks; update assess_execution (and the other similar branches at the other occurrences) to not return early: when self.unsafe_mode is True, still run the existing shell/AST/write confirmation checks (the functions that detect subprocess.run/os.system/exec and AST-only or file-write hazards) and aggregate their warnings with the result of is_dangerous_operation(code) before constructing and returning Decision; also expand is_dangerous_operation if needed to include delete-only and shell/AST-specific detections so that Decision(True, warnings) always includes the shell/AST/write warnings even in unsafe_mode. Ensure unique identifiers mentioned: assess_execution, self.unsafe_mode, is_dangerous_operation, and Decision are used to locate and fix the logic.
116-125:⚠️ Potential issue | 🟠 MajorNarrow deletion matching to filesystem APIs.
These checks still key on bare
.remove()/remove(, so ordinary in-memory code likeitems.remove("x")ormy_set.remove("x")gets blocked in SAFE mode and flagged as dangerous in UNSAFE mode. Please constrain the matcher to filesystem-specific owners/imports such asos.remove,Path.unlink, andshutil.rmtree.Also applies to: 183-198, 294-310
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/safety_manager.py` around lines 116 - 125, The current AST checks block any call named remove/unlink/rmtree even on in-memory objects; narrow them to filesystem APIs by verifying the call target before appending reasons: for ast.Attribute cases only flag when the attribute owner is a filesystem module/class (e.g., node.func.attr in ["remove","unlink","rmtree"] AND node.func.value is an ast.Name with id "os" or "shutil", or is a pathlib Path expression like an ast.Attribute/ast.Name that resolves to "Path" or "pathlib.Path"); similarly for getattr obfuscation only flag when the getattr target (node.args[0]) is an ast.Name/ast.Attribute that refers to "os", "shutil", or "Path"/"pathlib.Path" (or their common import aliases). Apply the same tightened logic to the other duplicated blocks around lines mentioned so the checks only match os.remove, shutil.rmtree, Path.unlink (including typical import aliasing), not arbitrary in-memory .remove calls.
328-345:⚠️ Potential issue | 🔴 CriticalArtifact export still follows symlinks and can clobber caller files.
Line 329 still defaults exports to
os.getcwd(), and Line 341 copies to a predictable destination withshutil.copy2(src, dst). A sandboxed symlink artifact or filename collision can still overwrite or read through host paths. Please export into an isolated directory, skip symlinks, and generate collision-safe names.Run this read-only check to confirm the current behavior and the relevant call sites:
#!/bin/bash python - <<'PY' import inspect import shutil print(inspect.signature(shutil.copy2)) PY rg -n "dest_dir = os\.getcwd\(\)|shutil\.copy2\(src, dst\)" libs/safety_manager.pyExpected result:
copy2showsfollow_symlinks=True, and the grep output points to the current default export directory and copy call.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/safety_manager.py` around lines 328 - 345, The export loop currently defaults dest_dir to os.getcwd(), follows symlinks and copies into predictable filenames via shutil.copy2(src, dst), which can clobber caller files; change the behavior so that when dest_dir is None you create an isolated temp directory (e.g., tempfile.mkdtemp/TemporaryDirectory) and always copy only regular files (skip entries where os.path.islink or stat.S_ISLNK is true and skip non-files), use a collision-safe destination name derived from the basename plus a unique suffix (uuid4 or tempfile.mktemp-like unique name) and call shutil.copy2 with follow_symlinks=False (or otherwise perform a safe read/write) to avoid following symlinks; ensure you join using os.path.basename(fname) to avoid path traversal and update the exported dict to map original fname to the actual unique dst path.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@libs/safety_manager.py`:
- Around line 74-82: The patterns in _WRITE_PATTERNS currently miss modes with a
'+' (e.g., "w+", "r+", "rb+"); update the regex entries in _WRITE_PATTERNS to
accept an optional '+' suffix for each write-related mode variant and similarly
extend any other mode checks to consider '+' (for example update the
mode-matching logic in the functions that validate modes—look for methods/vars
that parse mode strings around the existing checks such as any helper named
is_write_mode, _is_mode_write, or the code blocks that use the other regex lists
at the later checks referenced); ensure all occurrences that test for 'w', 'a',
'x', and 'r' modes also accept an optional '+' and binary/text suffixes so
open(..., "r+"), "rb+", "w+", "a+" and similar are detected as write-capable.
---
Duplicate comments:
In `@libs/safety_manager.py`:
- Around line 159-163: The unsafe_mode branch in assess_execution prematurely
returns a Decision and thus skips the shell/AST/write hazard checks; update
assess_execution (and the other similar branches at the other occurrences) to
not return early: when self.unsafe_mode is True, still run the existing
shell/AST/write confirmation checks (the functions that detect
subprocess.run/os.system/exec and AST-only or file-write hazards) and aggregate
their warnings with the result of is_dangerous_operation(code) before
constructing and returning Decision; also expand is_dangerous_operation if
needed to include delete-only and shell/AST-specific detections so that
Decision(True, warnings) always includes the shell/AST/write warnings even in
unsafe_mode. Ensure unique identifiers mentioned: assess_execution,
self.unsafe_mode, is_dangerous_operation, and Decision are used to locate and
fix the logic.
- Around line 116-125: The current AST checks block any call named
remove/unlink/rmtree even on in-memory objects; narrow them to filesystem APIs
by verifying the call target before appending reasons: for ast.Attribute cases
only flag when the attribute owner is a filesystem module/class (e.g.,
node.func.attr in ["remove","unlink","rmtree"] AND node.func.value is an
ast.Name with id "os" or "shutil", or is a pathlib Path expression like an
ast.Attribute/ast.Name that resolves to "Path" or "pathlib.Path"); similarly for
getattr obfuscation only flag when the getattr target (node.args[0]) is an
ast.Name/ast.Attribute that refers to "os", "shutil", or "Path"/"pathlib.Path"
(or their common import aliases). Apply the same tightened logic to the other
duplicated blocks around lines mentioned so the checks only match os.remove,
shutil.rmtree, Path.unlink (including typical import aliasing), not arbitrary
in-memory .remove calls.
- Around line 328-345: The export loop currently defaults dest_dir to
os.getcwd(), follows symlinks and copies into predictable filenames via
shutil.copy2(src, dst), which can clobber caller files; change the behavior so
that when dest_dir is None you create an isolated temp directory (e.g.,
tempfile.mkdtemp/TemporaryDirectory) and always copy only regular files (skip
entries where os.path.islink or stat.S_ISLNK is true and skip non-files), use a
collision-safe destination name derived from the basename plus a unique suffix
(uuid4 or tempfile.mktemp-like unique name) and call shutil.copy2 with
follow_symlinks=False (or otherwise perform a safe read/write) to avoid
following symlinks; ensure you join using os.path.basename(fname) to avoid path
traversal and update the exported dict to map original fname to the actual
unique dst path.
🪄 Autofix (Beta)
✅ Autofix completed
|
Note Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it. Fixes Applied SuccessfullyFixed 1 file(s) based on 1 unresolved review comment. Files modified:
Commit: The changes have been pushed to the Time taken: |
Fixed 1 file(s) based on 1 unresolved review comment. Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
… escape Critical #1 — assess_execution: block ALL absolute host-path access (reads included), not just writes/deletes. The is_path_access branch previously fell through to '# READ → ALLOWED', letting any code read /etc/passwd, /etc/hosts, ~/.ssh/id_rsa, etc. Now any code touching a host absolute path is rejected unconditionally in SAFE mode. Critical #2 — export_artifacts: harden against symlink + overwrite attacks. - Default dest_dir is now a fresh tempfile.mkdtemp(), NOT os.getcwd() (which let sandbox code plant a symlink → overwrite arbitrary host files) - Skip symlinks in source dir (os.path.islink guard) - Only copy regular files (os.path.isfile guard) - Collision-safe destination naming (fname_1.ext, fname_2.ext …) - shutil.copy2(..., follow_symlinks=False) — never follow symlinks on copy Minor — dangerous_patterns: add shutdown/reboot/mkfs/dd to block destructive system commands that were missing from the list. Minor — AST _ast_check: narrow .remove()/.unlink()/.rmtree() block to known-dangerous call targets (os, shutil, pathlib, Path) to eliminate false positives on e.g. list.remove() or dict.pop().
The _is_host_absolute_path check was blocking ALL absolute path references — including pure open(..., 'r') reads. This caused test_allows_read_only_absolute_path to fail. Fix: Only block absolute paths when a write operation is also present (_has_write_operation). Pure read-only access to absolute paths is permitted in SAFE mode. Sensitive system paths (/etc, /proc, /sys, /root) remain unconditionally blocked regardless of read/write mode.
test_blocks_write_function_with_absolute_path passes:
f = open('C:\\data.txt', 'r')
f.write('data')
open() uses mode 'r' so _WRITE_PATTERNS had no match, and the code
slipped through as allowed. Add \.write\s*\( to _WRITE_PATTERNS so
that any bare file-handle .write() call is caught regardless of the
open() mode used.
- Fix 1+5: Add system-destructive cmds (shutdown/reboot/mkfs/dd/diskpart/format) to safe-mode delete_patterns block; unify into shared _DESTRUCTIVE_PATTERNS constant to eliminate duplication between assess_execution & is_dangerous_operation - Fix 2: execute_command timeout handler now calls _kill_process_group() instead of process.kill() so child processes don't survive timeout - Fix 3: Replace _PYTHON_CODE_PATTERNS regex heuristic with ast.parse()-based detection in execute_script() — bash loops no longer misroute to Python - Fix 4: Remove nested any() bug in test_safety_manager_blocks_os_remove_* assertion (outer generator expression always evaluated to True) - Fix 6: Rename test_safety_manager_allows_relative_file_delete to test_safety_manager_blocks_relative_file_delete to match actual policy
…FE_EXECUTION attr
Bug #1 — _WRITE_PATTERNS: r"\.write\s*\(" was too broad, blocking sys.stdout.write(), buf.write(), socket.write() etc. Replaced with a file-handle scoped pattern that only matches write() preceded by a variable that was opened via open(), i.e. requires an assignment context. For simplicity and correctness, removed the bare .write() catch and rely on the open()-mode patterns + AST check instead. Bug #2 — shell_patterns "bash" substring check: plain `in` match on "bash" fires on any identifier containing the substring (e.g. "rehash", "bashful"). Replaced all shell_patterns string-in checks with re.search() using \b word-boundary anchors. Bug #3 — _DESTRUCTIVE_PATTERNS r"\bremove\(": fires on list.remove(), set.remove(), dict.pop() etc. — any method named remove(). The AST check already handles os/shutil/pathlib .remove() correctly. Replaced with r"\bos\.remove\s*\(" so only the real filesystem call is caught at regex level. Also tightened r"\bdelete\b" to r"\bdelete\s+\S" to avoid false-positives on SQL DELETE statements used as string literals in data-analysis code.
Bug 1 (test_blocks_write_function_with_absolute_path):
open('C:\\data.txt', 'r') + f.write('data') was allowed because:
- 'r' mode is not in _WRITE_PATTERNS (correct)
- bare .write( was removed in last session (correct for global block)
Fix: add r"\.write\s*\(" to _WRITE_ON_HANDLE_PATTERNS and check it
ONLY when an absolute path open() is already present in the code.
Bug 2 (test_safety_manager_allows_relative_file_delete):
r"import os\nos.remove('temp.txt')" — raw string so \n is literal
backslash+n. The char before 'o' in 'os.remove' is 'n' (word char)
so \b fails and the pattern never fires.
Fix: drop \b prefix from the os.remove pattern — the dot is already
a sufficient anchor since 'os.remove' cannot appear inside another
identifier."
…figFilesFromPR The test suite's TestNewConfigFilesFromPR.test_claude_sonnet_4_6_config_has_correct_model and the routing-matrix test both look for configs/claude-sonnet-4-6.json. The file existed only as claude-4-6-sonnet.json (wrong naming convention). Adding the canonically-named file with the correct model value.
…['\""] with ['\"] in single-quoted raw strings The patterns used r"...['\""...]" which caused the inner bare `"` to prematurely terminate the outer double-quoted string, producing E999 SyntaxError: invalid syntax at line 74. Fix: switch every affected pattern to a single-quoted raw string r'...' so the quote character class is written as ['"] without ambiguity. All regex semantics are identical — only the Python string delimiter changed.
…, fix unsafe execution timeout
…lease.sh - code_interpreter.py: Replace `python -c` with temp .py file to fix watchdog/timeout crash when executing complex code (plotly charts, pip subprocess calls). Temp file is cleaned up in finally block. - interpreter_lib.py: Add `/unsafe` slash command to toggle unsafe execution mode at runtime. Disables sandbox, timeout, and resource limits when active. Banner reflects current mode. - build_release.sh: Update to latest version with improved branch detection, version regex validation, and gh release target flag."
2c01653 to
a9ff30f
Compare
…/haseeb-heaven/code-interpreter into feature/code-sandbox-security-v3
…tore sandbox toggle alias, add subprocess security delegation, and increase SAFE mode MAX_TIMEOUT to 300s for more robust long‑running code execution
… alias, delegate subprocess security, and extend safe‑mode timeout for long‑running tasks.
v3.2.1 (2026-04-07)
Summary by CodeRabbit
New Features
--unsafeflag to allow confirmed write/delete operationsBug Fixes
Documentation
Tests
Chores